Namespace and storage management application infrastructure for use in management of resources in a storage system environment

ABSTRACT

A namespace and storage management (NSM) application includes an infrastructure configured to enable efficient management of resources in a storage system environment. The NSM application executes on a NSM console and interacts with an NSM server to integrate namespace management and storage management in the storage system environment. The NSM server, in turn, interacts with one or more remote agents installed on host machines in the environment to convey application programming interface (API) function calls that enable remote management of the resources.

CROSS-REFERENCE TO RELATED APPLICATION

The present application is a continuation of commonly assigned copendingU.S. patent application Ser. No. 11/414,593, which was filed on Apr. 28,2006, by Steven R. Klinkner et al., for a Namespace and StorageManagement Application Infrastructure for use in Management of Resourcesin a Storage System Environment, which is related to the followingcommonly assigned U.S. patent application Ser. No. 11/384,804 titled,System and Method for Integrating Namespace Management and StorageManagement in a Storage System Environment, filed Mar. 20, 2006, whichapplications are hereby incorporated by reference.

FIELD OF THE INVENTION

The present invention relates to storage systems, and more specifically,to an application infrastructure for use in management of resources in astorage system environment.

BACKGROUND OF THE INVENTION

A storage system typically comprises one or more storage devices intowhich information may be entered, and from which information may beobtained, as desired. The storage system includes an operating systemthat functionally organizes the system by, inter alia, invoking storageoperations in support of a storage service implemented by the system.The storage system generally provides its storage services through theexecution of software modules, such as processes. The storage system maybe implemented in accordance with a variety of storage architecturesincluding, but not limited to, a network-attached storage environment, astorage area network and a disk assembly directly attached to a clientor host computer. The storage devices are typically disk drivesorganized as a disk array, wherein the term “disk” commonly describes aself-contained rotating magnetic media storage device. The term disk inthis context is synonymous with hard disk drive (HDD) or direct accessstorage device (DASD).

The storage system may be further configured to operate according to aclient/server model of information delivery to thereby allow manyclients to access information stored on the system. In this model, thestorage system may be embodied as file server executing an operatingsystem, such as the Microsoft® Windows™ operating system (hereinafter“Windows operating system”). Furthermore, the client may comprise anapplication executing on an operating system of a computer that“connects” to the server over a computer network, such as apoint-to-point link, shared local area network, wide area network, orvirtual private network implemented over a public network, such as theInternet. Each client may request the services of the server by issuingstorage access protocol messages (in the form of packets) to the serverover the network. By supporting a plurality of storage (e.g.,file-based) access protocols, such as the conventional Common InternetFile System (CIFS) and the Network File System (NFS) protocols, theutility of the server is enhanced.

To facilitate client access to the information stored on the server, theWindows operating system typically exports units of storage, e.g.,(CIFS) shares. As used herein, a share is equivalent to a mount point orshared storage resource, such as a folder or directory that storesinformation about files or other directories served by the file server.A Windows client may access information in the directory by mounting theshare and issuing a CIFS protocol access request that specifies auniform naming convention (UNC) path to the share. The UNC path orpathname is an aspect of a Windows networking environment that defines away for a client to refer to a unit of storage on a server. The UNCpathname is prefixed with the string \\ to indicate resource names on anetwork. For example, a UNC pathname may comprise a server name, a share(directory) name and a path descriptor that collectively reference aunit of storage or share. Thus, in order to access the share, the clienttypically requires knowledge of the specific physical location (i.e.,the identity) of the server exporting the share.

Instead of requiring the client to provide the specific identity of thefile server exporting the share, it is desirable to only require alogical pathname to the share. That is, it is desirable to provide theclient with a globally unique pathname to the share without reference tothe file server. The conventional Distributed File System (DFS)namespace service provides such a solution in a Windows environmentthrough the creation of a namespace that removes the specificity ofserver identity. DFS is well-known and described in DCE 1.2.2 DFSAdministration Guide and Reference, 1997, which is hereby incorporatedby reference. As used herein, a namespace is a view of shared storageresources (such as shares) from the perspective of a client. The DFSnamespace service is generally implemented using one or more DFS serversand distributed components in a network.

Using the DFS service, it is possible to create a unique pathname (inthe form of a UNC pathname) for a storage resource that a DFS servertranslates to an actual location of the resource (share) in the network.However, in addition to the DFS namespace provided by the Windowsoperating system, there are many other namespace services provided byvarious operating system platforms, including the NFS namespace providedby the conventional Unix® operating system. Each service constructs anamespace to facilitate management of information using a layer ofindirection between a file server and client accessing a shared storageresource (share) on the server. For example, a share may be connected or“linked” to a link point (link in DFS terminology or a mount point inNFS terminology) to hide the machine specific reference to the share. Byreferencing the link point, the client can automatically accessinformation on the storage resource of the specific machine. This allowsan administrator (user) to store the information on any server in thenetwork by merely providing a reference to the information (or share).However, these namespaces are typically services created onheterogeneous server platforms, which leads to incompatibility andnon-interoperability with respect to management of the namespaces by theuser. For example, the DFS namespace service is generally limited toWindows-based operating system platforms, whereas the NFS namespaceservice is generally limited to Unix-based operating system platforms.

The Virtual File Manager (VFM™) developed by NuView, Inc. and availablefrom Network Appliance, Inc., (“NetApp”) provides a namespace servicethat supports various protocols operating on various file serverplatforms, such as NetApp filers and DFS servers. The VFM namespaceservice is well-known and described in VFM™ (Virtual File Manager)Reference Guide, Version 4.0, 2001-2003, and VFM™ (Virtual File Manager)Getting Started Guide, Version 4.0, 2001-2003.

A large-scale storage system environment may include many (e.g.,hundreds of) storage systems hosting resources embodied as, e.g.,storage and namespace objects. As used herein, namespace objects includenamespaces, link points and shares, whereas storage objects includeservers, files, and disks. In order to perform namespace and storagemanagement in such a large-scale storage system environment, it isdesirable to provide a management application executing on a managementconsole that enables users to manage multiple storage system resourcesefficiently, e.g., as one or more groups of managed objects. Inaddition, it is desirable to provide a management application thatallows users to efficiently operate on these groups of objectssimultaneously.

Often, users are required to provide appropriate credentials (e.g., username and password) in order to access the storage system resources forpurposes of performing operations thereon. Conventional managementconsoles are generally limited in their abilities to dynamically acquiresuch credentials when needed, i.e., “on-demand”. As a result, the usercredentials are typically previously acquired and stored for use by theconsoles, thereby requiring expensive persistent storage capabilities onthe management consoles. Accordingly, it is further desirable to providea management application executing on a management console that enablesusers to provide their credentials as needed (on-demand) when accessingstorage system resources.

SUMMARY OF THE INVENTION

The disadvantages of the prior art are overcome by providing a namespaceand storage management (NSM) application having an infrastructureconfigured to enable efficient management of resources in a storagesystem environment. The NSM application executes on a NSM console andinteracts with an NSM server to integrate namespace management andstorage management in the storage system environment. The NSM server, inturn, interacts with one or more remote agents installed on hostmachines in the environment to convey application programming interface(API) function calls (hereinafter “APIs”) that enable remote managementof the resources. Notably, the NSM application (and NSM console)cooperates with the NSM server and remote agent to create an integratedmanagement framework that provides an underlying infrastructure thatsupports various heterogeneous storage access protocols within a single,logical namespace service.

According to one aspect of the invention, the NSM applicationinfrastructure comprises one or more client presentation layers disposedover a library layer that is embodied as graphical user interface (GUI)and API components. To facilitate support for various clientpresentation layers, the library layer is apportioned into a GUI toolkitconfigured to produce reusable GUI components and an API toolkitconfigured to produce reusable API components that, in the illustrativeembodiment, are directed to namespace and storage management functions.The API toolkit handles thread management as well as API processing andcommunication with the NSM server, whereas the GUI components and theirimplementations are handled by the client presentation layer and GUItoolkit.

According to another aspect of the invention, the API toolkit comprisesan API client layer, a managed object layer, a thread pool layer and acredential handling layer. The API client layer illustratively embodiesan API library configured to generate APIs used to interact with the NSMserver. The managed object layer illustratively provides a plurality oftemplates that describes data and behavior associated with instances ofmanaged objects, as well as interfaces that enable operations on thoseobjects. The thread pool layer utilizes a plurality of worker threads toprocess an operation that is apportioned among the threads intomanageable APIs directed to the managed objects. The credential handlinglayer provides, among other things, a credential collection functionthat collects user credentials needed to access the resources“on-demand”.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and further advantages of the invention may be betterunderstood by referring to the following description in conjunction withthe accompanying drawings in which like reference numerals indicateidentical or functionally similar elements:

FIG. 1 is a schematic block diagram of an exemplary storage systemenvironment that may be advantageously used with the present invention;

FIG. 2 is a schematic block diagram of an exemplary namespace andstorage management server that may be advantageously used with thepresent invention;

FIG. 3 is a schematic block diagram of an integrated managementframework that may be advantageously used with the present invention;

FIG. 4 is a schematic block diagram illustrating an architecturalinfrastructure of a namespace and storage management (NSM) applicationin accordance with the present invention;

FIG. 5 is a schematic block diagram of an application programminginterface (API) toolkit of the NSM application in accordance with thepresent invention; and

FIG. 6 is a flowchart illustrating operation of the NSM application inaccordance with the present invention.

DETAILED DESCRIPTION OF AN ILLUSTRATIVE EMBODIMENT

A. Storage System Environment

FIG. 1 is a schematic block diagram of an exemplary storage systemenvironment 100 that may be advantageously used with the presentinvention. The storage system environment comprises a plurality ofstorage systems configured to provide storage services relating toinformation stored on storage devices, such as disks 140. The storagesystems include file servers 130 executing operating systems such as,e.g., the Microsoft® Windows™ operating system (hereinafter “Windowsoperating system”), the Unix® operating system and the NetApp® DataONTAP™ operating system available from Network Appliance, Inc.,Sunnyvale, Calif. One or more clients 110 may connect to the fileservers over a computer network 120, such as a point-to-point link,shared local area network, wide area network, or virtual private networkimplemented over a public network, such as the Internet.

Each client 110 may comprise an application executing on an operatingsystem of a general-purpose computer that interacts with the fileservers 130 in accordance with a client/server model of informationdelivery. That is, the client may request the services of a server, andthe server may return the results of the services requested by theclient, by exchanging packets over the network 120. The client may issuepackets including storage (e.g., file-based) access protocols, such asthe Common Internet File System (CIFS) protocol or Network File System(NFS) protocol, over the Transmission Control Protocol/Internet Protocol(TCP/IP) when accessing information in the form of, e.g., files anddirectories.

To facilitate client access to the information stored on the server, afile server 130 executing, e.g., the Windows operating system typicallyexports units of storage, e.g., (CIFS) shares. A client 110 may accessinformation of the share by mounting the share and issuing a CIFSprotocol access request that specifies a uniform naming convention (UNC)path to the share. Although the illustrative embodiment is directed to aWindows file server that is accessed using the CIFS protocol, theinvention described herein is not so limited and may also apply to otherservers 130, such as Unix file servers and NetApp filers, that areaccessible using other file access protocols, such as the NFS protocol.The client connects to file server 130 when accessing the share,typically by specifying the IP address (or identity) of the file server.

However, instead of requiring that the client know the specific identityof the server in order to access the share, a namespace server 160, suchas the conventional Distributed File System (DFS) server, provides anamespace service that removes the specificity of server identity. Usingthe DFS service, a unique pathname (in the form of a UNC pathname) iscreated for the share that the DFS server translates to an actuallocation of the share in the network. For example, assume a clientapplication issues a CIFS request to access information of a sharestored on a file server 130 in the environment 100. The request ispassed through various layers of the client's operating systemincluding, e.g., a CIFS redirector. Before issuing a CIFS packetdirected to a UNC pathname specified by the client application, theredirector contacts the DFS server to determine whether there is adifferent, actual pathname needed to access the information. If so, theDFS server returns that actual pathname to the redirector, which thenuses that pathname to create the CIFS packet for transmission over thenetwork. The DFS namespace service thus provides a level of indirectionto a share that resides on a file server 130 in the environment 100.

Although the illustrative embodiment is directed to a DFS namespaceservice provided by the Windows operating system executing on a DFSserver, the invention described herein is not limited and may also applyto other namespace services, such as the NFS namespace provided by theUnix operating system executing on a conventional Name InformationService (NIS) server. Each service constructs a namespace to facilitatemanagement of information using a layer of indirection between a fileserver 130 and client 110 accessing a shared storage resource (share) onthe server. However, these namespace services are created onheterogeneous server platforms, which lead to incompatibility andnon-interoperability with respect to management of different namespacesby, e.g., an administrator (user).

In the illustrative embodiment, a namespace and storage management (NSM)server 200 is provided that integrates namespace management and storagemanagement in storage system environment 100. The NSM server 200includes an integrated management framework that provides an underlyinginfrastructure that supports various heterogeneous storage accessprotocols within a single, logical namespace service. To that end, theNSM server cooperates with a NSM console 150 and a remote agent 170 tocreate the integrated management framework. The integrated managementframework, in turn, allows the NSM server 200 to interact with anynamespace server, such as a DFS server configured to handle CIFSprotocol requests or a NIS server configured to handle NFS protocolrequests, directed to namespace services.

FIG. 2 is a schematic block diagram of an exemplary namespace andstorage management (NSM) server 200 that may be advantageously used withthe present invention. The NSM server illustratively comprises aprocessor 210, a memory 220, a network adapter 230 and a storage adapter240 interconnected by a system bus 250. The memory 220 may comprisestorage locations addressable by the processor and adapters for storingsoftware programs, i.e., specific sets of ordered operations, and datastructures associated with the invention. The processor and adaptersmay, in turn, comprise processing elements and/or logic circuitryconfigured to execute the programs and manipulate the data structures.In particular, the storage adapter 240 comprises the mechanical,electrical and signaling circuitry needed to connect the server tostorage devices, such as disks 245. Similarly, the network adapter 230comprises the mechanical, electrical and signaling circuitry needed toconnect the server 200 to, e.g., the NSM console 150 and remote agent170.

An operating system 225, portions of which is typically resident inmemory 220 and executed by the processing elements, functionallyorganizes the server by, inter alia, invoking operations in support ofstorage services implemented by the server. In the illustrativeembodiment, the operating system is preferably the Windows operatingsystem, although it is expressly contemplated that any appropriateoperating system, such as the Unix operating system, may be enhanced foruse in accordance with the inventive principles described herein. Theoperations invoked by the operating system are illustratively namespaceand storage operations in support of an integrated management framework300 provided by the server. It will be apparent to those skilled in theart that other processor and memory means, including various computerreadable media, may be used for storing and executing programinstructions pertaining to the invention described herein.

B. Integrated Management Framework

The NSM server 200 generally provides its storage services through theexecution of software modules, such as processes. These services areillustratively implemented as separately-scheduled processes (and/ordaemons) that execute in user space of operating system 225 to providethe integrated management framework 300. As used herein, a processrefers to an instance of a program being executed by, e.g., theprocessor and a thread is an instance of a sequence of the process'sprogram code. FIG. 3 is a schematic block diagram of the integratedmanagement framework 300 that may be advantageously used with thepresent invention. The integrated management framework 300 provides alogical namespace service that is based on extensions to underlyingstorage management technology and other technological components.Notably, these extensions are embodied as library functionality.

In the illustrative embodiment, the underlying storage managementtechnology is embodied as DataFabric® Manager (DFM) technology availablefrom Network Appliance, Inc., Sunnyvale, Calif. and described inDataFabric® Manager 3.2 Administration Guide, 1994-2004, which is herebyincorporated by reference. Broadly stated, the DFM technology comprisesa set of coordinating processes, including an application is programminginterface (API) server 310, a scheduler 320, an Event daemon 330, aMonitor 340 including a Discovery daemon 350 and a database interface(DBI) 360 configured to interact with an embedded database 370. Thecapabilities of these processes are extended to create the integratednamespace and storage management framework 300 through the addition ofNSM library functionality 380. The NSM library 380 is illustrativelyimplemented as various library modules, each of which provides namespaceand storage management actions embodied as APIs.

The integrated management framework 300 exposes a set of interfaces,e.g., an API interface that is used by the NSM console 150 and a commandline interface (CLI 315), used by a communication protocol.Illustratively, the communication protocol is embodied as an XML overHTTP mechanism for APIs, primarily because of its ability to describedata conveyed over the network, as well as the transport, withoutdescribing the actual storage access protocol. An example of acommunication protocol that may be advantageously used with the presentinvention is the ZAPI protocol available from Network Appliance, Inc.Such a protocol is also easily extensible and flexibly agnostic tobinary formats of specific servers and clients to thereby operateefficiently in a heterogeneous environment.

In the illustrative embodiment, the integrated management framework 300facilitates configuration and management of pathnames in the logicalnamespace. That is, the underlying infrastructure of the frameworkallows a user to manage various pathnames exported by heterogeneousnamespace service and protocol implementations within the logicalnamespace. This aspect of the framework involves creation of a layer ofabstraction that presents to a client a notion of the logical namespacethat is accessible via a particular storage access protocol. As noted,the various heterogeneous namespace services are not interoperable withrespect to user management; the integrated framework 300 extractslogical “views” (e.g., pathnames) from the namespaces exported by theheterogeneous services, stores those pathnames in database 370, and thenconfigures the pathnames so that they are accessible in the logicalnamespace via the storage access protocol.

In storage and networking environments, some operations can only beexecuted if a user (such as a system administrator) is logged into aserver on which the operations are to be performed, e.g., the NSMserver. In other words, a remote operation that is to be invoked on theserver 200 can only be performed if the user is logged into the server.The NSM console 150 and remote agent 170 of the integrated managementframework 300 address this problem. Illustratively, the NSM console 150is a component of the framework that executes an NSM application 400having a JAVA-based interface and the remote agent 170 is a softwaremodule installed on a host machine, such as a server 160, which managesa particular namespace, e.g., a DFS server. Using the communicationprotocol, the NSM console issues APIs to the NSM server which, in turn,issues APIs that are invoked on the remote agent to perform actionsrequested by the user.

Essentially, the remote agent 170 is a thin web server that includes“programmatic glue” to enable installation of plug-in modules(“plug-ins”). An example of a plug-in is a namespace management module175 that allows remote access and management of data (e.g., pathnames)by the NSM server 200. Notwithstanding the particular namespace, theintegrated management framework 300 involves installation of a remoteagent 170 on a server 160 that manages the particular namespace andinteracts with the NSM server 200 via APIs exposed by the remote agent.

Illustratively, the remote agent 170 is installed on a machine thathosts a DFS server of a Windows/DFS environment. The namespacemanagement plug-in module 175 may, in turn, be installed on the remoteagent to enable communication with the NSM server 200 when accessingnamespace information stored on the DFS server. Specifically, the NSMserver issues APIs in connection with the communication protocol toinstruct the remote agent 170 to perform certain actions. For example,in order to construct or change pathnames in the logical namespace, auser interacts with the NSM console 150 to access that namespace by, ineffect, invoking a set of APIs on the NSM server 200 which, in turn,invokes a set of APIs on the remote agent 170. Once the NSM server hascooperated with the DFS server to construct the pathnames, those serversare no longer involved in client data accesses.

It should be noted that the underlying directory data structures used inthe logical namespace reside on the namespace server 160 providing theparticular namespace service, such as a DFS server. That is, thenamespace server 160 implements the basic namespace; the NSM server 200provides a service for configuring that namespace. To that end, the NSMserver 200 cooperates with the remote agent 170 to acquire a copy of thenamespace directory structures from the namespace server 160 and storesthe copy on the embedded database 370. The NSM server then cooperateswith the NSM console 150 to allow a user to manipulate (configure) thecopy. Once configuration is complete, the server 200 cooperates with theremote agent to “push” (store) the manipulated copy of directorystructures back to the namespace server 160.

The Monitor 340 is illustratively embodied as a multi-threaded processhaving a collection of individual monitor threads, each of which isscheduled by the scheduler 320. The Monitor 340 cooperates with theremote agent 170 to communicate with one or more machines/devices in thestorage system environment 100 (using SNMP, RSH, etc) to collect anytype of storage or data/namespace object information (e.g., volumeand/or qtree information, as well as information about namespaceservers) available in the environment and, to that end, functions as adata collector. Illustratively, each monitor thread is configured tocollect information about a particular object in the environment.Extensions to the Monitor described herein are directed to namespace andagent management, each of which is illustratively embodied as anindividual monitor thread.

As data is collected, the Monitor 340 determines whether it needs tonotify any processes of some resulting condition. For example, anindividual monitor thread may detect a threshold and the occurrence ofsome type of event. As used herein, an event is a synchronousnotification with a severity attribute. By configuring thresholds, theNSM server 200 can determine when an event occurs. If a threshold isexceeded, the Monitor 340 communicates with the Event daemon 330 tonotify it of that event. Thus, the Monitor 340 is configured to detectcertain events and, in response, generate event notifications to theEvent daemon 330. The Event daemon 330 then determines what (if any)further action is needed (e.g., send an SMTP alert, an email, an SNMPtrap) and records the event in the embedded database 370.

The embedded database 370 is illustratively implemented as a Sybaserelational database that exports a set of Structured Query Language(SQL) function calls that enable storage/retrieval of data, such asnamespace objects, to/from the database. A schema of the database isconfigured to efficiently capture characteristics of a namespaceindependent of the storage access protocol, such as NFS or CIFS. Thedatabase 370 is illustratively organized as a plurality of tables, eachof which can be accessed by processes within the NSM server. The DBI 360is illustratively embodied as a SQL interface to the database and itsassociated tables, although it will be understood to those skilled inthe art that the DBI may be embodied as any other type of databaseprotocol interface depending on the actual implementation of thedatabase, e.g., an Oracle database.

The API server 310 is illustratively embodied as a multi-threadedprocess that is configured to implement the APIs used by the NSM console150 to access the database 370 and manipulate information storedthereon. The API server 310 also implements the communication protocolAPIs used to interact with the remote agent 170. Accordingly, extensionsto the API server 310 are primarily directed to those APIs required tomanipulate the database and the remote agent. For example, APIs areinvoked by a user (via the NSM console 150) to manipulate (modify,update) the embedded database 370. The API server 310 updates thedatabase with respect to user configuration requests and the Monitor 340periodically queries (polls) the database for any state/informationchange. Depending upon the change, the API server may cooperate with theMonitor to instruct data collection from the remote agent 170. Afterreceiving the collected information from the remote agent, the Monitor340 updates the database and the Event daemon 330.

C. NSM Application Infrastructure

The present invention is directed to the NSM application 400 having aninfrastructure configured to enable efficient management of resources ina storage system environment. The NSM application 400 executes on theNSM console 150 and interacts with the NSM server 200 to integratenamespace management and storage management in storage systemenvironment 100. The NSM server, in turn, interacts with one or more isremote agents 170 installed on host machines in the environment toconvey API function calls (hereinafter “APIs”) that enable remotemanagement of the resources. In an alternate embodiment, however, theNSM application can interact directly with machines hosting the remoteagents. As noted, the NSM application (and NSM console) cooperates withthe NSM server and remote agent to create the integrated managementframework 300 that provides an underlying infrastructure that supportsvarious heterogeneous storage access protocols within a single, logicalnamespace service.

In the illustrative embodiment, the NSM application 400 is a“lightweight” JAVA-based client application executing on the NSM console150, which is embodied as a computer having, e.g., a Windows, Linux orSolaris operating system platform. The NSM application 400 isillustratively a single process that, as a JAVA-based application, ischaracterized by a plurality of threads that can be easily ported tomulti-platform instantiations. However, the NSM application is“lightweight” in that it does not require a persistent database andsubstantially all operations involve APIs executed on other hostmachines and servers.

FIG. 4 is a schematic block diagram illustrating an architecturalinfrastructure of the NSM application 400 in accordance with the presentinvention. The NSM application infrastructure comprises one or moreclient presentation layers 410 disposed over a library layer 420 that isembodied as graphical user interface (GUI) and application programminginterface (API) components. The client presentation layers 410 may bedirected to various GUI level presentations, such as storage managementand namespace management. To facilitate support for various clientpresentation layers, the library layer 420 is apportioned into a GUItoolkit 430 configured to produce reusable GUI components, such asdialog boxes (dialogs), and an API toolkit 500 configured to producereusable API components that, in the illustrative embodiment, aredirected to namespace and storage management functions. As describedherein, the API toolkit 500 handles thread management, as well as APIprocessing and communication with the NSM server 200, whereas the GUItoolkit 430 handles the GUI components and their implementations.

Broadly stated, the NSM application 400 spawns a GUI thread that is usedby the is client presentation layer 410 and GUI toolkit 430 to provideGUI components that enable a user to input, e.g., commands for managingselected objects on various servers 130. The client presentation layer410 translates the command to an operation and provides the operation,an operation failure handler and a group of managed objects to the APItoolkit 500. The API toolkit 500 processes the operation to generate aplurality of APIs and spawns a plurality of threads (“worker threads”)to manage the APIs directed to the managed objects of the group. As usedherein, the term “API” denotes a collection of methods/functions(“routines”) exposed by, e.g., processes of the integrated managementframework 300 to which other processes issue calls. When a problem(failure) arises, the API toolkit 500 calls the operation (API) handler,which may result in the presentation of a GUI component, e.g., a dialog155, by the client presentation layer 410. Notably, the behavior of theAPI in light of a failure is specified by the client presentation layer;as such, the API toolkit 500 handles API failures according tofunctionality provided by the client presentation layer 410.

FIG. 5 is a schematic block diagram of the API toolkit 500 of the NSMapplication 400 in accordance with the present invention. The APItoolkit 500 comprises an API client layer 520, a managed object layer540, a thread pool layer 560 and a credential handling layer 580,wherein the layers interact and communicate using conventionalintra-process message passing techniques. The API client layer 520illustratively embodies an API library 522 configured to generate APIsused to communicate with the NSM server 200. These APIs areillustratively contained within XML tags and transported to the NSMserver in accordance with the communication protocol.

The managed object layer 540 illustratively provides a plurality oftemplates, e.g., JAVA classes, that describes data and behaviorassociated with instances of managed objects, as well as interfaces thatenable operations on those objects as, e.g., programming constructs. Asused herein, a managed object may represent any namespace and/or storageresource that has a backing API on a host machine or server; that is,the managed object may represent any resource for which the machine orserver exports APIs for management. To that end, the managed objectscomprise managed namespace objects 542 representing, e.g., namespaces,link points and shares, and managed storage objects 544 representing,e.g., servers, volumes, files and disks. The managed object layer 540interacts with the API client layer 520 to package these managed objectswithin the XML tags.

The thread pool layer 560 utilizes a plurality of worker threads 562 toprocess an operation that is apportioned among the threads intomanageable APIs directed to the managed objects. This layer 560 of theAPI toolkit 500 also provides a thread pooling function that controlsthe number of simultaneously executing worker threads so as not toconsume substantial processing resources (e.g., CPU) on the NSM console150. Often a user may select a group of managed objects to perform anoperation thereon. In such a situation, the user is interested in anoverall result of the operation across the group, rather than for eachindividual object. Accordingly, the thread pool layer further maintainsa data structure embodied as a result tracker 565 that is adapted tomonitor the progress of the APIs and provide a completion notification(e.g., success or failure) summarizing the thread operation results onthe objects. This layer also provides functionality adapted to forcetime-outs on APIs that would otherwise be subject to arbitrary networkdelays due to, e.g., non-responsive servers.

The credential handling layer 580 provides a credential collectionfunction that collects user credentials 586 (e.g., user name andpassword) needed to access the resources “on-demand” using, e.g., anauthentication callback interface 582 to a GUI component, e.g., a dialog155. This layer further maintains the collected credentials 586 in alocal cache 585 (note that the NSM console 150 does not persistentlystore credentials). Caching of the credentials allows the credentialhandling layer 580 to determine whether previously collected credentialscan be used to satisfy pending requests by worker threads 562 forpermission to access the resources in accordance with, e.g., acredential sharing function. For example, the credential sharingfunction may illustratively attempt to resolve all pending credentialrequests for each server using the cached credentials and prompt a useronly when a request cannot be resolved, i.e., when the cachedcredentials do no include the necessary credential to perform a pendingrequest. Alternatively, the credential sharing function may use one setof cached credentials per server to resolve the requests and only promptthe user on failure to resolve. In addition, the credential handlinglayer 580 provides a locking mechanism 588 that limits the number ofdialogs 155 that may be displayed on the NSM console 150 at a time whencollecting credentials.

Specifically, in the illustrative embodiment, the credential handlinglayer 580 coordinates access by the worker threads 562 to thepresentation of a dialog 155 used to collect and share credentials. Thatis, credential collection involves the use of the dialog that may bedisplayed by the threads to collect credentials. Each worker thread 562that manages a failed API may initiate display of the dialog 155 on theNSM console 150 to request the credentials from the user in accordancewith the on-demand credential collection function. However, assume theoperation provided by the client presentation layer 410 is decomposedinto, e.g., hundreds of APIs managed by corresponding worker threads562. If all of these APIs fail simultaneously, there may be hundreds ofdialogs 155 appearing on the NSM console at a time. To obviate thissituation, the locking mechanism 588 of the credential handling layer580 ensures that only a limited number of worker thread 562s interactswith the client presentation layer 410 to control display of a limitednumber of dialogs 155, e.g., one, at a time on the NSM console 150. Inaddition, credential sharing allows the user to enter the credentialsinto the dialog 155 and, for those credentials that may be used toaccess managed objects on various servers 130, share those credentialsamong the worker threads 562.

FIG. 6 is a flowchart illustrating operation of the NSM application inaccordance with the present invention. The procedure 600 starts at Step602 and proceeds to Step 604 where a user interacts with the NSMapplication executing on the NSM console to initiate performance of anoperation directed to a group of managed objects, e.g., units ofstorage, hosted by a plurality of servers. Assume that the operationmanipulates the units of storage (e.g., CIFS shares) exported byoperating systems executing on the servers. In Step 606, the clientpresentation layer provides the operation and an indication of themanaged objects to the API toolkit. In Step 608, the managed objectlayer and thread pool layer cooperate to apportion (decompose) theoperation into a plurality of APIs directed to the managed objects and,in Step 610, spawn one or more worker threads to manage the APIs. InStep 612, the API client layer formats and sends the APIs within XMLtags to the NSM server using the communication protocol. The NSM serverillustratively processes the APIs in a manner that, e.g., translatesthose APIs into additional APIs that initiate actions by one or moreremote agents directed to the managed objects.

In Step 614, the NSM server transmits the APIs to the remote agent and,in Step 616, the remote agent forwards APIs to the servers hosting themanaged objects. Note that the remote agent communicates with anappropriate storage access protocol server (e.g., a CIFS server) of theoperating system on each server using (e.g., Win32 API) remote procedurecalls (RPCs) over the network. In Step 618, a determination is made asto whether user credentials are provided to permit access to the managedobjects. Note also that this determination may be needed because thecredentials are simply unavailable (empty) or because an attempt toexecute an API against a server using available credentials may fail(due to bad/stale credentials). If the credentials are (successfully)provided, the servers execute the APIs directed to the managed objectsin Step 620 and the procedure ends at Step 634. However, if usercredentials are not provided (or if an API attempt fails because thecredentials provided are bad/stale), in Step 622 a failure notificationis returned to the NSM application (via the NSM server) for eachattempted RPC communication between the agent and server, indicatingfailure of the API due to missing user credentials.

In Step 624, the credential handling layer of the API toolkit cooperateswith the client presentation layer to prompt the user, on-demand, toprovide its credentials (permission) to access the managed objects onthe server. Furthermore, the credential handling layer ensures that,e.g., only one worker thread interacts with the client presentationlayer to control display of only one dialog at a time on the NSM console(Step 626). Illustratively, control of the dialog is not passed off toanother worker thread that is waiting on credential collection until thecurrent thread completes such collection. In Step 628, the user entersits credentials to the dialog and, in Step 630, the credential handlinglayer shares, where appropriate, those credentials among worker threadswaiting on the dialog. In Step 632, the API client layer formats andsends the credentials (with APIs, if needed) to the servers (via the NSMserver and remote agents) to enable execution of the APIs on the managedobjects and the procedure returns to Step 618.

While there has been shown and described an illustrative embodiment ofan NSM application having an infrastructure configured to enableefficient management of resources in a storage system environment, it isto be understood that various other adaptations and modifications may bemade within the spirit and scope of the invention. For example, thepresentation of a dialog 155 used to collect/share credentials is anillustrative implementation of the credential sharing function of thecredential handling layer 580. Here, the actual behavior of thisfunction is specified by the presentation layer 410 and is independentof the library 420. In an alternate implementation, the worker threads562 may be directed to a file containing the credentials rather thanprompting a user with the dialog 155. This alternate behavior is usefulfor automating execution of the application when interaction with theuser is not possible (e.g., automated testing).

The foregoing description has been directed to specific embodiments ofthis invention. It will be apparent, however, that other variations andmodifications may be made to the described embodiments, with theattainment of some or all of their advantages. For instance, it isexpressly contemplated that the procedures, processes, layers and/ormodules described herein may be implemented in hardware, software,embodied as a computer-readable medium having executable programinstructions, firmware, or a combination thereof. Accordingly thisdescription is to be taken only by way of example and not to otherwiselimit the scope of the invention. Therefore, it is the object of theappended claims to cover all such variations and modifications as comewithin the true spirit and scope of the invention.

What is claimed is:
 1. A system configured to enable management ofresources in a storage system environment, the system comprising: amanagement server configured to interact with a plurality of namespaceservers, the management server including a processor and a memory forstoring operations executable by the processor; a management applicationexecuting on a management console computer, the management applicationhaving an infrastructure including at least one client presentationlayer disposed over a library layer apportioned into at least oneapplication programming interface (API) toolkit configured to: handleAPI processing and communication with the management server and producereusable components directed to namespace and storage managementfunctions, the API toolkit cooperating with the client presentationlayer to prompt a user for user credentials, to be provided on-demand,to perform the operation upon the resources in the storage system,employ a thread pool layer configured to utilize the worker is threadsto process the operation apportioned among the worker threads intomanageable APIs directed to the resources; and a result trackermaintained by the thread pool layer, the result tracker configured tomonitor progress of the manageable APIs and provide a completionnotification summarizing thread operation results on the resources. 2.The system of claim 1 wherein the management application is furtherconfigured to receive an operation directed to the resources wherein theoperation is managed by a plurality of worker threads, the managementapplication further configured to interact with the management server tointegrate namespace management and storage management in the storagesystem environment.
 3. The system of claim 1 wherein the API toolkitcomprises an API client layer that is embodied as an API libraryconfigured to generate APIs used to interact with the management server.4. The system of claim 3 wherein the resources include instances ofmanaged objects and wherein the API toolkit further comprises a managedobject layer configured to provide a plurality of templates thatdescribe data and behavior associated with the managed objects, as wellas interfaces that enable operations on those objects.
 5. The system ofclaim 1 wherein one worker thread interacts with the client presentationlayer to control display of only one dialog at a time on the managementconsole to obtain the user credentials from the user and the one workerthread to share the user credentials among other worker threads, wherethe user credential from the one worker thread and the other workerthreads is sent to at least a first server and a second server, with thepurpose of performing the operation upon the resources.
 6. The system ofclaim 1 wherein the API toolkit comprises a credential handling layerconfigured to provide an on-demand credential collection function thatcollects the user credentials needed to access the resources.
 7. Thesystem of claim 1 wherein the management application is a namespace andstorage management (NSM) application, the management console computer isa NSM console and the management server is a NSM server.
 8. A method forenabling management of resources in a storage system environment, themethod comprising: executing a management application on a managementconsole computer to initiate performance of an operation directed to agroup of managed objects, the executed management application enablinginteraction with a management server to integrate namespace managementand storage management in the storage system environment, wherein thegroup of managed objects are units of storage hosted by a plurality ofservers; decomposing the operation into a plurality of routines directedto the group of managed objects; spawning a plurality of threads on themanagement application to manage the plurality of routines; determiningthat user credentials are not provided to permit access by a user to thegroup of managed objects; in response to determining that the usercredentials are not provided, prompting the user for the usercredentials, to be provided on-demand, to enable execution of theplurality of routines upon the group of managed objects on the pluralityof servers; sending the user credentials, from the one thread and theother threads, to the plurality of servers, hosting the group of managedobjects, the user credentials configured to enable execution of theplurality of routines on the group of managed objects; utilizing anapplication programming interface (API) toolkit configured to handle APIprocessing and communication with the management server, wherein the APItoolkit includes a thread pool layer configured to utilize the pluralityof threads to process the plurality of routines that is apportionedamong the plurality of threads into manageable APIs directed to thegroup of managed objects; and maintaining a result tracker by the threadpool layer, the result tracker configured to monitor progress of themanageable APIs and provide a completion notifications summarizingthread operation results on the managed objects.
 9. The method of claim8 wherein the plurality of threads require the user credentials and onethread of the plurality of threads controls display of only one dialogat a time on the management console computer to obtain the usercredentials from the user, wherein the one thread shares the usercredentials among other threads, the other threads waiting on the dialogfor the credentials, with the purpose of executing the plurality ofroutines upon the group of managed objects
 10. The method of claim 8wherein the management application comprises a namespace and storagemanagement (NSM) application and wherein the management console computercomprises an NSM console.
 11. The method of claim 8 further comprising:sending manageable APIs to an NSM server configured to translate themanageable APIs into additional APIs that initiate action by a remoteagent directed to the managed objects; and forwarding the additionalAPIs to servers hosting the managed objects.
 12. The method of claim 11further comprising: returning a failure notification to the NSMapplication indicating failure of an API due to user credentials thatwere not provided, before prompting the user for the user credentials.13. The method of claim 8 wherein the managed object comprises one ormore shares.
 14. The method of claim 8, further comprising: sending theroutines to the management server, the management server including aprocessor configured to process the routines; translating, by themanagement server, the routines into additional routines, the additionalroutines configured to initiate an action by a remote agent directed tothe managed objects; and forwarding the additional routines to theplurality of servers hosting the managed objects.
 15. An apparatusconfigured to enable management of resources in a storage systemenvironment, the apparatus comprising: means for executing a managementapplication on a management console computer to initiate performance ofan operation directed to a group of managed objects, the executedmanagement application enabling interaction with a management server tointegrate namespace management and storage management in the storagesystem environment, wherein the group of managed objects are units ofstorage hosted by a plurality of servers; means for decomposing theoperation into a plurality of routines directed to the group of managedobjects; means for spawning a plurality of threads on the managementapplication to manage the plurality of routines; means for determiningthat the user credentials are not provided to permit access by a user tothe group of managed objects; is means for prompting the user for theuser credentials, to be provided on-demand, to enable execution of theplurality of routines upon the group of managed objects on the pluralityof servers; means for sending the user credentials, from the one threadand the other threads, to the plurality of servers, to enable executionof the plurality of routines on the group of managed objects; means forutilizing an application programming interface (API) toolkit configuredto handle API processing and communication with the management server,wherein the API toolkit includes a thread pool layer configured toutilize the plurality of threads to process the plurality of routinesthat is apportioned among the plurality of threads into manageable APIsdirected to the group of managed objects; and means for maintaining aresult tracker by the thread pool layer, the result tracker configuredto monitor progress of the manageable APIs and provide a completionnotifications summarizing thread operation results on the managedobjects.
 16. The apparatus of claim 15 wherein the plurality of thethreads require the user credentials and one thread controls display ofonly one dialog at a time on the management console computer to obtainthe user credentials from the user in response to determining that theuser credentials are not provided, wherein the one thread shares theuser credentials among other threads, the other threads waiting on thedialog for the credentials, with the purpose of executing the pluralityof routines upon the group of managed objects.
 17. The apparatus ofclaim 15, further comprising: means for sending the routines to themanagement server, the management server including a processorconfigured to process the routines; means for translating the routinesinto additional routines, the additional routines configured to initiatean action by a remote agent directed to the managed objects; and meansfor forwarding the additional routines to the plurality of servershosting the managed objects.
 18. A non-transitory computer readablestorage medium containing executable program instructions executed by aprocessor, comprising: program instructions that execute a managementapplication on a management console computer to initiate performance ofan operation directed to a group of managed objects, the executedmanagement application enabling interaction with a management server tointegrate namespace management and storage management in the storagesystem environment; program instructions that decompose the operationinto a plurality of routines directed to the group of managed objects;program instructions that spawn a plurality of threads on the managementapplication to manage the plurality of routines; program instructionsthat determine user credentials are not provided to permit a user toaccess the group of managed objects; program instructions that prompt auser for the user credentials, to be provided on-demand, to perform theplurality of routines upon the group of managed objects on the pluralityof servers; program instructions that send the user credentials, fromthe one thread and the other threads, to the plurality of servers, toenable execution of the plurality of routines on the group of managedobjects; program instructions that utilize an application programminginterface (API) toolkit configured to handle API processing andcommunication with the management server, wherein the API toolkitincludes a thread pool layer configured to utilize the plurality ofthreads to process the plurality of routines that is apportioned amongthe plurality of threads into manageable APIs directed to the group ofmanaged objects; and program instructions that maintain a result trackerby the thread pool layer, the result tracker configured to monitorprogress of the manageable APIs and provide a completion notificationssummarizing thread operation results on the managed objects.
 19. Thenon-transitory computer readable storage medium of claim 18 wherein onethread controls display of only one dialog at a time on the managementconsole computer to obtain the user credentials, wherein the one threadshares the user credentials among other threads, the other threadswaiting on the dialog for the credentials, with the purpose ofperforming the plurality of routines upon the group of managed objects.